Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link Python module to Cantera shared library #1429

Merged
merged 28 commits into from
Feb 3, 2023

Conversation

speth
Copy link
Member

@speth speth commented Jan 24, 2023

Changes proposed in this pull request

The main purpose of this PR is to modify the implementation of ExtensibleRate so that the core Cantera library (libcantera) does not need to depend on a specific Python installation at compilation / installation time. Achieving this required some fairly far-reaching changes, though I think they will all have long-term benefits.

  • The PythonExtensionManager class and associated Cython functions have been moved out of libcantera into a separate shim, libcantera_python that is loaded on demand. This new library is the one that links to libpython and loads the Cantera Python module.
  • Because a user C++ program and the Cantera Python module (loaded via libcantera_python) both depend on libcantera, libcantera needs to be loaded as a shared library to avoid having two copies of the library with separate state (which would interfere instantiating user defined types that are only instance with one copy of ReactionRateFactory, for example). Multiple versions of Python are supported (not at the same time!) by having multiple versions of libcantera_python, e.g. libcantera_python3_11.so and libcantera_python3_10.so.
  • On Windows, linking C++ applications and the Cantera Python module to libcantera_shared requires "exporting" the complete C++ interface. Due the the sheer number of methods that need to be exported, rather than the standard approach investigated in Export C++ library symbols for core methods #1298, this PR uses the static library to generate a list of symbols to be exported from the DLL.
  • The functions to load the libcantera_python library on-demand are OS-specific. Rather than use these low-level functions, the Boost.DLL library is used for its higher-level, cross-platform capabilities. While Boost.DLL is header-only, by default it depends on Boost.Filesystem, which has a compiled component. However, since C++17 introduced std::filesystem (derived from Boost.Filesystem), Boost.DLL provides an option to use std::filesystem. Therefore, in order to avoid introducing a dependence on the compiled components of Boost, this PR bumps Cantera's compiler requirement to the C++17 standard.

There are also a few more minor changes that ended up coming along for the ride:

  • Add using directives within the Cantera namespace for some of the most-used standard library types: string, vector, map, function, pair, unique_ptr. I figure map<string, string> is more readable than std::map<std::string, std::string>.
  • Bump vendored versions of fmt and Eigen.
  • Move some function definitions out of the header files, especially in the case of the Factory classes.

If applicable, fill in the issue number this pull request is fixing

Checklist

  • The pull request includes a clear description of this code change
  • Commit messages have short titles and reference relevant issues
  • Build passes (scons build & scons test) and unit tests address code coverage
  • Style & formatting of contributed code follows contributing guidelines
  • The pull request is ready for review

@codecov
Copy link

codecov bot commented Jan 24, 2023

Codecov Report

Merging #1429 (3dd702d) into main (c0ae563) will increase coverage by 0.21%.
The diff coverage is 74.79%.

@@            Coverage Diff             @@
##             main    #1429      +/-   ##
==========================================
+ Coverage   70.73%   70.94%   +0.21%     
==========================================
  Files         378      369       -9     
  Lines       55141    55201      +60     
  Branches    18164    18176      +12     
==========================================
+ Hits        39006    39165     +159     
+ Misses      13626    13574      -52     
+ Partials     2509     2462      -47     
Impacted Files Coverage Δ
include/cantera/base/ExtensionManager.h 27.27% <0.00%> (-15.59%) ⬇️
include/cantera/base/ExtensionManagerFactory.h 100.00% <ø> (ø)
include/cantera/base/global.h 80.95% <ø> (ø)
include/cantera/cython/funcWrapper.h 74.56% <ø> (ø)
include/cantera/kinetics/Arrhenius.h 93.93% <ø> (+1.25%) ⬆️
include/cantera/kinetics/BlowersMaselRate.h 97.43% <ø> (+2.08%) ⬆️
include/cantera/kinetics/ChebyshevRate.h 100.00% <ø> (+1.72%) ⬆️
include/cantera/kinetics/Custom.h 100.00% <ø> (+50.00%) ⬆️
include/cantera/kinetics/PlogRate.h 100.00% <ø> (ø)
include/cantera/kinetics/TwoTempPlasmaRate.h 100.00% <ø> (+5.26%) ⬆️
... and 72 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@speth speth force-pushed the python-link-shared branch 7 times, most recently from f6080c6 to 1fc2b7a Compare January 24, 2023 21:44
@speth speth marked this pull request as draft January 25, 2023 15:38
@speth speth force-pushed the python-link-shared branch 18 times, most recently from 8485cb3 to 508967a Compare February 1, 2023 04:56
This is a prerequisite to using Boost.DLL with the std::filesystem library.
This eliminates the need to directly link the Cantera shared library
or Cantera applications to libpython.
This version properly exports all necessary DLL symbols, eliminating the
need to embed it separately in the Cython module.
10.15 or newer is required for C++17 support
This requires copying the relevant MinGW runtime libraries into the
Python module.
Trying this to understand why this step can sometimes take
upwards of 8 minutes.
This is important because extensions can only be loaded correctly when
Cantera is linked as a shared library.
When the Python module is linked to the Cantera shared library,
it is possible that a user has multiple incompatible versions of
the library installed. This checks that the Cantera version and
Git commit are the same when importing the Python module, to avoid
crashes or erroneous behavior due to ABI mismatches.
Copy link
Member

@ischoegl ischoegl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Export C++ symbols in dynamic Windows libraries
2 participants